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[Name of the Document] 

Specification 
[Title of the Invention] 

Image distribution system, image 
distribution method and recording medium 
there for 

[What is claimed is] 
[Claim 1 ] 

An image distribution system in a 
virtual space system composed of terminal 
apparatuses respectively provided at 
plural users and a server apparatus 
connected to such plural terminal 
apparatuses through a communication 
channel for constructing a virtual space 
for distributing an image among the 
terminal apparatuses, wherein each of the 
terminal apparatuses comprises: 

image obtaining means for obtaining 
an image of the user; 

transmission means for transmitting 
the image, obtained by said image 
obtaining means, to said server 

apparatus ; 

reception display means for receiving 
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and displaying the image transmitted from 
said server apparatus; 

designation means for designating the 
position of the user in said virtual 
space; and 

control means for controlling said 
image obtaining means according to the 
position of the user in said virtual 
space . 
[Claim 2 ] 

An image distribution system 

according to claim 1, wherein said image 
obtaining means further includes image 
compression means for compressing the 
image data; and 

said control means controls the 
compression parameter of said 

compression means according to said user 
position in said virtual space. 
[ C 1 a im 3 ] 

An image distribution system 

according to claim 1, wherein said image 
obtaining means includes size conversion 
means for converting the size of the 
obtained image and cut-out means for 



2 



cutting out a predetermined area from the 
obtained image; and 

wherein said control means selects 
the output of said size conversion means 
or said cut-out means according to said 
user position .in said virtual space. 
[Claim 4 ] 

An image distribution system 

according to claim 1, wherein said image 
obtaining means isn eludes image pickup 
means for converting an optical image 
into an electrical signal and image 
pickup control means for controlling an 
area and a direction of the image pickup 
of said image pickup means; and 

said control means controls the image 
pickup area of said image pickup means 
through said image pickup control means 
according to said user position in said 
virtual space. 
[Claim 5 ] 

An image distribution system 

according to claim 1, wherein said image 
obtaining means includes plural image 
pickup means for converting optical 
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images into electrical signals; and 
said control means selects one of the 
outputs of said plural image pickup means 
according to said user position in said 
virtual space. 
[Claim 6 ] 

An image distribution system in a 
virtual space system composed of terminal 
apparatuses respectively provided at 
plural users and a server apparatus 
connected to such plural terminal 
apparatuses through a communication 
channel for constructing a virtual space 
for distributing an image among the 
terminal apparatuses, wherein each of the 
terminal apparatuses comprises: 

image obtaining means for obtaining 
an image of the user; 

transmission means for transmitting 
the image, obtained by said image 
obtaining means, to said server 

apparatus ; 

reception display means for receiving 
and displaying the image transmitted from 
said server apparatus; and 
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designation means for designating the 
position of the user in said virtual 
space; and said server apparatus 

comprises : 

image process means for processing 
the image transmitted from each of said 
terminal apparatuses; 

distribution means for distributing 
the image, processed by said image 
process means, to each of said terminal 
apparatuses; and 

control means for controlling said 
image process means according to the 
position of the user in said virtual 
space . 
[Claim 7 ] 

An image distribution system 

according to claim 6 , wherein said image 
process means includes recompression 
means for recompressing image data; and 
wherein said control means controls the 
compression parameter at the 

recompression, according to the user 
position in said virtual space. 
[Claim 8 ] 
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An image distribution system 

according to claim 6 , wherein said image 
obtaining means includes size conversion 
means for converting the size of the image 
and cut-out means for cutting out a 
predetermined area from the image; and 
said control means selects the output of 
said size conversion means or said 
cut-out means according to said user 
position in said virtual space. 
[Claim 9 ] 

An image distribution method in a 
virtual space system composed of terminal 
apparatuses respectively provided at 
plural users and a server apparatus 
connected to such plural terminal 
apparatuses through a communication 
channel for constructing a virtual space 
for distributing an image among the 
terminal apparatuses, wherein each 

terminal apparatus is characterized in 
that it transmits user images 

corresponding to a user position in said 
virtual space to said server apparatus 
and receives and displays the images 
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transmitted from said server apparatus. 
[Claim 10] 

An image distribution method 

according to claim 9 , wherein each of said 
terminal apparatuses transmits the user 
image, compressed with a compression rate 
according to said user position in said 
virtual space, to said server apparatus. 
[Claim 11] 

An image distribution method 

according to claim 9 , 

wherein each of said terminal 
apparatuses applies either of the image 
size conversion and the predetermined 
area cutting-out to the obtained image 
according to said user position in said 
virtual space, and transmits the image to 
said server apparatus. 
[Claim 12] 

An image distribution method 

according to claim 9, wherein each of said 
terminal apparatuses includes image 
pickup means for converting an optical 
image into an electrical signal and image 
pickup control means for controlling the 
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area and direction of the image picking 
up by said image pickup means, and 
controls the area of the image picking up 
by said image pickup means according to 
said 1 user position in said virtual space. 
[ C 1 a i m 1 3 ] 

An image distribution method 

according to claim 9 , wherein each of said 
terminal apparatuses includes plural 
image pickup means for converting an 
optical image into an electrical signal, 
and selects one of the outputs of said 
plural image pickup means according to 
said user position in said virtual space, 
and transmits the image obtained by the 
selected image pickup means to said 
server apparatus. 
[Claim 14] 

An image distribution method in a 
virtual space system composed of terminal 
apparatuses respectively provided at 
plural users and a server apparatus 
connected to such plural terminal 
apparatuses through a communication 
channel for constructing a virtual space 
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for distributing an image among the 
terminal apparatuses, wherein 

each of said terminal apparatuses 
obtains an image of the user and transmits 
the image to said server apparatus; 

said server apparatus processes the 
image transmitted from each of the 
terminal apparatuses according to the 
position of the user in said virtual space 
and distributes the image to each of said 
terminal apparatuses; and 

each of said terminal apparatuses 
receives and displays the image 

transmitted from said server apparatus. 
[Claim 15] 

An image distribution method 

according to claim 14, 

wherein said server apparatus 

recompresses the image transmitted from 
each of said terminal apparatus with a 
recompression parameter according to the 
user position in said virtual space and 
distributes the image to each of said 
terminal apparatuses. 
[Claim 16] 
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An image distribution method 

according to claim 14, 

wherein said server apparatus applies 
either of image size conversion and 
predetermined area cutting-out to the 
image transmitted from each of said 
terminal apparatuses according to said 
user position in said virtual space and 
distributes the image to each of said 
terminal apparatuses . 
[Claim 17] 

A memory medium which is 

characterized in that it stores a 
software program for executing the image 
distribution method according to any one 
of claims 9 to 16. 



[Detailed Description of the Invention] 
[0001] 

[Industrial Field of the Invention] 

The present invention relates to an 
image distribution system, a method and 
a program therefor, and more particularly 
to an image distribution system in a 
virtual system formed on a computer 
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network, a method and a recording medium 
therefor . 

[0002] 

[Prior Art] 

In organizations such as business 
firms, associations or public bodies, it 
is customary that an office space is 
secured by possession or by renting as the 
work space for employed laborers 

(hereinafter called workers) to be 
engaged in office works, marketing works 
or technical works not requiring a large 
facility, and the workers arrive and 
gather at the office space by public 
transportation or by private cars by a 
work start time and execute group work 
within the predetermined work time. Such 
office space is equipped with telephones, 
copying machines, facsimiles, computers, 
computer networks etc. for improving the 
efficiency of various works. 

[0003] 

In the history of human beings, the 
habit of such group work is a relatively 
new phenomenon adopted for the efficient 
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operation of the factories since the 
industrial revolution. In recent years, 
the drawbacks resulting from such group 
work such as congestion in commuting and 
air pollution caused by the increase of 
the private cars have become apparent, 
while the group work is becoming not 
essential for the organizations and for 
their workers as a result of availability 
of the communication infrastructure such 
as internet and of various communication 
technologies. Therefore the attention 
is being directed to the concept of 
dispersed work in which the workers 
belonging to an organization executes 
their jobs at their homes or in arbitrary 
locations close thereto whereby the 
objective of the organization is attained 
as a whole. 
[0004] 

In general, for realizing the 
dispersed work in an organization such as 
a business firm, there is adopted a method 
of utilizing a room of the home of each 
worker as a working space (called home 
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office), connecting plural home offices 
in distant locations by a communication 
channel and utilizing home -use 

communication terminals such as a 
telephone set and a facsimile apparatus 
and a communication application system. 
In this manner there is secured the 
communicating means among the workers. 
The communication channel can be composed, 
for example, of a public telephone 
network, an ISDN network or an internet 
exclusive line, and the communication 
application system can be composed of an 
e-mail system, a www (worldwide web) or 
a video conference system. 
[0005] 

The worker may also utilize a 
satellite office or a telecottage 
prepared by the organization or the local 
community to which the worker belongs or, 
in case of a sales job or a maintenance 
job, the seat of a private car or a public 
transportation vehicle, to be used for 
visiting the customer, as a temporary 
office (called mobile office). 
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[0006] 

[Problems to be Solved by the Invention] 
For the purpose of smoothly realizing 
such dispersed work, there have been 
proposed various systems. Fig. 18 shows 
a client image in a multi-spot video 
conference provided by White Pine 
Software Inc., U. S. This system 

realizes, on personal computers, a 
real-time conference utilizing images 
among workers in distant locations. 
[0007] 

The mutual exchange of the images 
among the dispersed workers through such 
system allows to realize so-called 
interactive awareness system thereby 
avoiding isolated or secluded feeling of 
the workers and stimulating active 
communication among the workers. 
[0008] 

However, since the purpose of use 
is different in case the video conference 
system is utilized as a main 

communication tool and in case it is 
utilized as an interactive awareness 
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system, the transmitted image results in 
various limitations in relation to the 
communication band and the convenience of 
use if the conventional system is applied 
for such purpose. More specifically, the 
user is required to adjust the camera 
setting and the compression parameter 
according to the function to be utilized. 
[ 0 0 0 9] 

An object of the present invention 
is, in order to resolve such drawbacks, 
to provide an image distribution system 
capable of constructing a virtual space 
system on a data network and mutually 
communicating an appropriate image 
according to the position of the user in 
the virtual space, and a method and a 
memory medium therefor. 
[0010] 

Incidentally, the following 

description, the "position in virtual 
space" in relation to the present 
invention means, in case of constructing 
a virtual office space for example in a 
network, a position in a virtual space 
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a server apparatus connected to such 
plural terminal apparatuses through a 
communication channel for constructing a 
virtual space for distributing an image 
among the terminal apparatuses, wherein 
it is characterized in that each of the 
terminal apparatuses comprises image 
obtaining means for obtaining an image of 
the user, transmission means for 

transmitting the image, obtained by the 
image obtaining means, to the server 
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apparatus, reception display means for 
receiving and displaying the image 
transmitted from the server apparatus, 
designation means for designating the 
position of the user in the virtual space, 
and control means for controlling the 
image obtaining means according to the 
position of the user in the virtual space. 
[0012] 

The image distribution system 

according to the present invention is 
also composed of terminal apparatuses 
respectively provided at plural users and 
a server apparatus connected to such 
plural terminal apparatuses through a 
communication channel for constructing a 
virtual space for distributing an image 
among the terminal apparatuses, wherein 
it is characterized in that each of the 
terminal apparatuses comprises image 
obtaining means for obtaining an image of 
the user, transmission means for 

transmitting the image, obtained by the 
image obtaining means, to the server 
apparatus, reception display means for 
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receiving and displaying the image 
transmitted from the server apparatus, 
and designation means for designating the 
position of the user in the virtual space, 
and the server apparatus comprises image 
processing means for processing images 
transmitted from each of the terminal 
apparatuses, distribution means for 
distributing processed images being 
processed by the image processing means, 
and control means for controlling the 
image processing means according to a 
user position inside the virtual space of 
a receiving terminal. 
[0013] 

The image distribution method 

according to the present invention is 
characterized by the use in a virtual 
space system composed of terminal 
apparatuses respectively provided at 
plural users and a server apparatus 
connected to such plural terminal 
apparatuses through a communication 
channel for constructing a virtual space 
for distributing an image among the 
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terminal apparatuses, wherein each 
terminal apparatus is characterized in 
that it transmits user images 

corresponding to a user position in said 
virtual space to said server apparatus 
and receives and displays the images 
transmitted from said server apparatus. 
[0014] 

The image distribution method 

according to the present invention is 
characterized by the use in a virtual 
space system composed of terminal 
apparatuses respectively provided at 
plural users and a server apparatus 
connected to such plural terminal 
apparatuses through a communication 
channel for constructing a virtual space 
for distributing an image among the 
terminal apparatuses, wherein each of 
said terminal apparatuses obtains an 
image of the user and transmits the image 
to said server apparatus; said server 
apparatus processes the image 

transmitted from each of the terminal 
apparatuses according to the position of 
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the user in said virtual space and 
distributes the image to each of said 
terminal apparatuses; and each of said 
terminal apparatuses receives and 

displays the image transmitted from said 
server apparatus. 
[0015] 

A software program for performing the 
above-mentioned image distribution 

system is stored in a memory medium of the 
present invention. 
[0016] 

[ Embodiments ] 

Now embodiments of the present 
invention will be described in detail 
with reference to the accompanying 
drawings . 
[0017] 

Fig. 1 is a schematic block 
diagram showing the configuration of a 
virtual office system constituting an 
embodiment of the present invention. In 
a main office 10 (for example the main 
office of an organization such as a 
business firm introducing the dispersed 
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office system), there is installed a host 
server apparatus 12 which is connected to 
a LAN (local area network) 14 in the main 
office 10. User terminal apparatuses 16, 
18 for the workers in the main office 10 
are realized by combining a computer 20, 
a user terminal apparatus software 22, a 
telephone set 24, a video camera 26 and 
other devices. The telephone 24 can be 
connected to the host server apparatus 12 
through a PSTN channel (public telephone 
channel) or an ISDN channel, and the 
computer 20 is connected to the host 
server apparatus 12 through the LAN 14. 
[0018] 

User terminal apparatuses 30, 32, 
34 for the workers in home offices are 
respectively realized by combining a 
computer 36, a user terminal apparatus 
software 38, a telephone set 40, a video 
camera 42, a virtual office display 
apparatus 44 and other devices. The 
telephone 40 can be connected to the host 
server apparatus 12 through a PSTN 
channel (public telephone channel) or an 
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ISDN channel. The virtual office display 
apparatus 4 4 is connected to the computer 
3 6 . 

[0019] 

Fig. 2 shows an example of the 
hardware configuration of the host server 
apparatus 12. A basic platform (BP) 12a 
is composed of a PC host server apparatus. 
A signal processing unit (SPU) 12b is a 
signal processing board composed for 
example of a high-performance DSP 

(digital signal processor). The SPU 12b 
is utilized for signal processing for 
example for image and voice at the server. 
The BP 12a is connected through a LAN board 
to the LAN 14 in the main office 10. Such 
configuration is already known and there 
may also be employed other 

configurations . 

[0020] 

Fig. 3 is a schematic block 
diagram showing the software 

configuration of the host server 
apparatus 12. The software installed in 
the host server apparatus 12 includes 
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software programs developed with the C++ 
language or the like and already existing 
software programs, and, as the OS 
(operating system) , there can be employed 
the Windows NT (trade name of Microsoft 
Inc., U. S . ) . In Fig. 3, there are shown 
a server manager 50, an SPU access library 
52, drivers 54, 56, a mail transmission 
unit 58, a DDL (dynamic link library) 60, 
a dynamic web server 62, a database 
connector 64, Windows NT (registered 
trademark of Microsoft Inc., U. S.) 66, 
and a database 68. 
[0021] 

Fig. 4 shows an example of the 
configuration of the user terminal 
apparatus 30. The computer 36 is 

composed, as already known, of a main body 
70, a mouse 72, a keyboard 74, a display 
76, a microphone 78, a speaker 80 and a 
modem (or a terminal adaptor) 82. The 
virtual office display apparatus 44 is 
composed of an image monitor apparatus 
such as a display and is connected to the 
main body 70 for displaying a virtual 
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office image. The video camera 32 is also 
connected to the main body 70. The mouse 
72, keyboard 74, display 76, microphone 
78, speaker 80 and modem 82 are connected 
to corresponding input/output terminals 
of the main body 70. 
[0022] 

Fig. 5 shows an example of 

installation of the devices shown in Fig. 
4, in a home office. In this example, the 
virtual office display apparatus 44 is 
installed in a position different from 
that of the display 76 of the computer 36 
to be used by the user for the works. 
[0023] 

In the following there will be 
explained the function of the present 
embodiment. For the purpose of 

simplicity, it is assumed that the users 
of the home offices have already started 
to work by completing the connection to 
the host server apparatus 12 utilizing 
the respective user terminal apparatuses 
3 0, 32, 34. 
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[0024] 

Fig. 6 shows an image, called 
office view, displayed on the virtual 
office display apparatus 4 4. There are 
shown an image 90 showing the state of work 
of another user, a work state data display 
portion 92 for displaying the work state 
data by text information, and private 
office space 94 of such user. These three 
displays in combination constitute a 
virtual private office of the user. Fig. 
6 shows nine private offices on a same 
image, but the number of such private 
offices can be larger or smaller. The 
space for displaying such private offices 
is called a private office area 96. A 
space between the private offices is a 
virtual corridor. The user work state 
image 90, displayed as a part of the 
private office, is taken by the camera 42 
provided in the user terminal apparatus 
30 to 34 used by such user. 
[0025] 

At the right-hand side of the private 
office area 96, there is provided a common 
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area 98, in which displayed are icons 100 
to 108 respectively representing a 
conference room A, a conference room B , 
a laboratory, a refreshing room and a 
video news room. 
[0026] 

When the worker is working, the 
virtual office display apparatus 4 4 
displays the office view as shown in Fig. 
6. When the user is positioned in the 
office area in the virtual space, the 
system functions as an interactive 
awareness system so that the exchanged 
image can be of such content as to show 
the outlined state of the user. 
[0027] 

Fig. 7 shows an example of the 
virtual office display image called 
conference room. When the user clicks a 
conference room icon 100 on the office 
view, the display shifts to the 

conference room on the virtual space as 
shown in Fig. 7. In a conference room 
display area 110 there are displayed 
images 112 of other participants and an 
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image 114 schematically showing a desk. 
An image 116 of the user is taken by the 
camera provided in the user terminal used 
by such user. A reference display area 
118 displays references to be used at the 
conference and registered in advance by 
the participants. 
[0028] 

Each participant executes the 
conference by looking at the image and the 
reference, utilizing unrepresented voice 
conference tool (also called voice chat 
tool) and character conference tool (also 
called text chat tool). 
[0029] 

The voice conference tool 

includes a voice conference client 
software (voice telephone conference 
software for TCP/IP network based on the 
ITU-T standard H.323) provided in the 
user terminal apparatus and a voice 
conference server software (multi-spot 
voice telephone conference software for 
TCP/IP network based on the ITU-T 
standard H.323) provided in the host 



27 



server apparatus, and, when these 
softwares are activated, the users can 
have a voice conference in multiple 
locations utilizing the microphone and 
the speaker provided in each user 
terminal apparatus. 
[0030] 

The text conference tool includes 
a text conference client software (text 
conference client software for TCP/IP 
network based on the IRC (internet relay 
chat) standard) provided in the user 
terminal apparatus and a text conference 
server software (multi-spot text 

conference server software for TCP/IP 
network based on the IRC conference) 
provided in the host server apparatus, 
and, when these softwares are activated, 
the users can have a text conference in 
multiple locations utilizing the 

keyboard of each user terminal apparatus 
and a text chat window displayed 
simultaneously on the display separately 
from the conference room image (Fig. 7). 
[ 0 0 3 1] 
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The voice conference tool, the text 
conference tool and the multi-user game 
tool can be based on the known 
technologies, and will not be explained 
further since they are not related with 
the present embodiment. 
[0032] 

When the user is present in the 
conference room, it is important, for 
attaining smooth communication, to 
clearly transmit the face of the 
participant, thereby enabling to 

recognize the expression of each 
participant . 
[0033] 

Fig. 8 is a flow chart showing the 
operations relating to the obtaining and 
displaying image data and image control 
in the user terminal apparatus. In the 
following there will be explained the 
image distribution control operation of 
the virtual office system of the present 
embodiment, with reference to the flow 
chart shown in Fig. 8. 
[0034] 
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The image taken by each video 
camera 4 2 is taken into the main body 70 
of the computer by a video input device 
thereof, as image data of QCIF format (176 
x 144 pixels) (SI). Then the position of 
the user in the virtual space is judged 
(S2), and, in case of the office view 
(namely in case the user is present in the 
virtual large office space), there is set 
a first compression parameter (S3), but, 
if otherwise, there is set a second 
compression parameter (S4). The 

compression parameter is assumed to be a 
quantization coefficient. More 

specifically, in case the user is present 
in the office view, namely in case the user 
utilizes the present system as an 
interactive awareness system, there is 
selected a large quantization 

coefficient (S3), but, in other cases, 
for example if the user is present in the 
conference room, there is selected a 
small quantization coefficient, and a 
face area is detected for example a method 
reported by Yokoo and Ogiwara, 
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"Extraction of plural face areas from a 
natural image utilizing a hereditary 
algorithm", Den-Gaku-Ron 117-C, 9 , pp. 
1 2 4 5 - 1 2 5 2 ( ( 1 9 9 7 ) and a smaller 

quantization coefficient is assigned to 
the face area ( S 4 ) . 
[0035] 

The image data are compression 
encoded by an image encoding method such 
as JPEG or H . 2 6 3 , utilizing the 

designated quantization coefficient ( S 5 ) . 
For example, in case of JPEG compression, 
the transformation coefficients obtained 
by color space conversion and DCT are 
quantized by the designated quantization 
coefficient and then compression by the 
Huffman encoding. The compressed image 
data are promptly transferred through the 
network to the host server apparatus 12 
( S 6 ) . 
[0036] 

The user terminal receives the 
images of other users from the host server 
apparatus 12 (S7), expands the image data 
of such images ( S 8 ) and displays them on 
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the display apparatus 4 4 ( S 9 ) . 
[0037] 

These operations are repeated 
until the user instructs the end of the 
process (S10) . 
[0038] 

In the present embodiment, the 
taken image is compressed with an 
automatically predetermined 
quantization coefficient, according to 
the position of the user in the virtual 
space. In case the user utilizes the 
interactive awareness function, the 
compression is executed with the large 
quantization coefficient to display 
other users as rough images. On the other 
hand, in case of the conference in the 
virtual conference room, the images of 
other users are displayed with a high 
definition. Therefore, in case of 

utilizing the interactive awareness 
function, the data amount of the images 
of other users is reduced so that the 
communication band can be utilized 
efficiently. Also high-definition 
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images can be exchanged in case of 
utilizing the communicating function 
principally such as the conference. It 
is also possible to show the face area with 
a higher definition' in case of utilizing 
the active communicating function, 
utilizing the face recognition 

technology . 
[0039] 

Such control of the compression rate 
according to the function corresponding 
to the position in the virtual space 
allows to transmit the optimum image 
while efficiently utilizing the 

communication band. 
[0040] 

In the example shown in Fig. 8, the 
image compression rate is switched 
according to the user position, but it is 
also possible to switch either to cut out 
a principal portion of the taken image or 
to reduce the size of the taken image 
according to the user position. Fig. 9 
shows a flow chart of such variation. 
[0041] 
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A field wider than the image to be 
actually transmitted is taken by the 
video camera 4 2,. and the data of such taken 
image are fetched into the main body 70 
of the computer by the video input device 
thereof ( S 1 1 ) . In this operation, there 
is for example taken a full scene 
indicating the state of the user. 
[0042] 

Then the position of the user in 
the virtual space is judged ( S 1 2 ) , and, 
in case the user is present in the office 
view (namely in case the user utilizes the 
present system as an interactive 
awareness system, the fetched image is 
linearly reduced to a size corresponding 
to the transfer rate of the communication 
channel (S13), but, in other cases, for 
example if the user is present in the 
conference room, an area principally 
containing the user face is cut out from 
the fetched image (S14). The cut-out 
area may be predetermined by the user, or 
the face area of the user may be 
automatically detected and cut out based 
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on the face recognition technology as 
explained in relation to Fig. 8. 
[0043] 

Figs. 10 show examples of image 
reduction (S13) and face cut-out (S14). 
Fig. 10(a) indicates a taken image, while 
Fig. 10(b) shows an image reduced to a 
desired size from the image shown in Fig. 
10(a) and Fig. 10(c) shows an image 
obtained by cutting out a face area of a 
desired size from that shown in Fig. 
10(a) . 
[ 0 0 4 4 ] 

Such reduced image (S13) or 
cut-out image (S14) is compression 
encoded by an image encoding method such 
as JPEG or H.263 ( S 1 5 ) , and the compressed 
image data are promptly transferred 
through the network to the host server 
apparatus 12 (S16). The user terminal 
receives the images of other users from 
the host server apparatus 12 ( S 1 7 ) , 
expands the image data of such images 
(S18) and displays them on the display 
apparatus 44 (S19). 
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[ 0 0 4 5 ] 

These operations are repeated 
until the user instructs the end of the 
process (S20) . 
[0046] 

The convenience of use can be 
improved since it is automatically 
selected either to reduce the image or to 
cut out the predetermined portion so as 
to enable recognition of the user face, 
according to the user position in the 
virtual space. For example in case the 
user utilizes the interactive awareness 
function, the full-scene images of other 
users are received as rough images. On 
the other hand, in case of the conference 
in the virtual conference room, other 
users are displayed by images principally 
composed of the faces thereof. Therefore, 
in case of utilizing the awareness mode, 
it is possible to understand the status 
of other user, and, in case of utilizing 
the function principally based on 
communication such as the conference, 
satisfactory communication can be 
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realized by high-definition images 
principally based on the faces. 
[0047] 

Such control of the transmitted 
image area according to the function 
corresponding to the position in the 
virtual space allows to realize 

transmission control of the optimum image 
according to the purpose of use. 
[0048] 

Fig. 11 is a flow chart showing a 
3rd example of the operations of the 
present embodiment. The user terminal 
judges the position of the user in the 
virtual space ( S 2 1 ) . In case the user is 
present in the office view (S21) namely 
in case the user utilizes the interactive 
awareness function, the zooming and the 
image taking direction of the camera 42 
are so controlled as to take a full scene 
indicating the state of the user (S22). 
In other cases (S21), the zooming and the 
image taking direction of the camera 42 
are so controlled as to take a narrow area 
principally containing the user face 
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(S23). The camera 4 2 can be composed of 
a camera in which the panning, tilting and 
zooming can be controlled externally. 
The image taking area may be 

predetermined by the user, or 

automatically selected. Particularly 
in case of taking an image principally 
consisting of the user face, the image 
taking area may be determined by 
automatically detecting the face area of 
the user utilizing the face recognition 
technology as already explained in 
relation to Fig. 8. 
[0049] 

The image is taken with thus set 
image taking area and is fetched into the 
main body ( S 2 4 ) . The fetched image is 
compression encoded by an image encoding 
method such as JPEG or H.263 (S25), and 
the compressed image data are promptly 
transferred through the network to the 
host server apparatus 12 (S26) . The user 
terminal receives the images of other 
users from the host server apparatus 12 
(S27), expands the image data of such 
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images (S28) and displays them on the 

display apparatus 4 4 (S29). 

[0050] 

These operations are repeated 
until the user instructs the end of the 
process (S30) . 
[0051] 

As explained in the foregoing, the 
image taking area is selected according 
to the user position in the virtual space, 
and the full-scene image is transmitted 
in case the user utilizes the interactive 
awareness function, but the image 
principally composed of the face is 
transmitted in other cases. Therefore it 
is possible to understand the state of 
other users in case of the awareness mode, 
and satisfactory communication is made 
possible by the high-definition image 
principally composed of the face in case 
of the function mainly for communication 
such as the conference. Also the process 
load is reduced since the image taking 
area can be selected solely by the camera 
control and does not require any 
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particular image processing. 
[0052] 

Such control of the image taking 
device according to the function 
corresponding to the position in the 
virtual space allows realizing 

transmission control of the optimum image 
according to the purpose of use. 
[0053] 

As shown in a plan view in Fig. 12, 
the single video camera 42 may be replaced 
by a video camera 42a positioned in front 
of the user and adapted to take an image 
principally composed of the user face, 
and a video camera 42b taking the full 
scene where the user is present. There 
are also shown a video switching unit 120 
for switching the outputs of the video 
camera 42a, 42b, a user terminal 122 and 
a user 124. The video switching unit 120 
is connected for example to a serial 
interface of the user terminal 122 and is 
controlled thereby. 
[0 0 5 4 ] 

Fig. 13 is a flow chart showing the 



40 



operations in case the two video cameras 
42a , 42b are provided as shown in Fig. 12. 
[0055] 

The user terminal 122 judges the 
position of the user in the virtual space 

( S3 1 ) , and switches the video switching 
unit 120 according to the result of the 
judgment, thereby selecting the camera 
42a or 42b to be used ( S32 ) , In case the 
user is present in the office view, namely 
in case the user utilizes the interactive 
awareness function, there is selected the 
camera 42b for taking the full scene 
indicating the state of the user. In this 
case, the full scene is taken from the back 
in order to reduce the psychological 
resistance of the user. On the other hand, 
in case the user is in the conference room, 
there is selected the camera 42a for 
taking the image of the narrow area 
principally composed of the user face. 

[0 0 5 6 ] 

The image taken with the selected 
video camera 42a or 42b is fetched into 
the main body 70 (S33) . The fetched image 
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is compression encoded by an image 
encoding method such as JPEG or H.263 

( S 3 4 ) , and the compressed image data are 
promptly transferred through the network 
to the host server apparatus 12 ( S 3 5 ) . 
The user terminal receives the images of 
other users from the host server 
apparatus 12 ( S 3 6 ) , expands the image 
data of such images (S37) and displays 
them on the display apparatus 4 4 ( S 3 8 ) . 

[0057] 

These operations are repeated until 
the user instructs the end of the process 
( S 3 9 ) . 
[ 0 0 5 8 ] 

Such selection of the cameras 42a, 
42b according to the user position in the 
virtual space allows to distribute the 
image showing the desired area. For 
example, the full scene of the user is 
distributed as a rough image in case the 
user utilizes the interactive awareness 
function, but the image principally 
showing the user face is displayed in case 
of the conference in the virtual 
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conference room. Therefore, it is 

possible to understand the state of other 
users in case of the awareness mode, and 
the images principally composed of faces, 
suitable for communication are exchanged 
in case of the function mainly for 
communication such as the conference. 
[0059] 

As explained in the foregoing, the 
load of signal processing can be reduced 
since the image taking area is controlled 
solely by the camera selection and do not 
require any particular signal processing 
Also the use of plural cameras increases 
the freedom in setting the image taking 
angle corresponding to the function to be 
used, and the convenience of use can be 
improved particularly in reducing the 
psychological burden of the user. 
[0060] 

In the following there will be 
explained an example of process according 
to the user position by the host server 
apparatus 12. Fig. 14 is a flow chart 
showing the operations of the user 
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present in the office view ( S 5 2 ) , the 
received image data' are partially 
expanded. For example, in case of the 
image data compressed with the JPEG 
method, the decoding is executed by the 
Huffman decoding and the inverse DCT to 
the DCT coefficients. Then the restored 
DCT coefficients are again quantized with 
a quantization coefficient larger than 
that at the compression encoding ( S 5 4 ) , 
and are compression encoded again by 
entropy encoding such as Huffman encoding 
(S55). The host server apparatus 12 

distributes the image data thus subjected 
to partial decoding and recompression to 
the user terminal through the network 
( S 5 6 ) . 
[0063] 

The user terminal receives the images 
of other users from the host server 
apparatus 12 (S44), expands the image 
data of such images (S45) and displays 
them on the display apparatus 44 (S46). 
[0064] 

The user terminal repeats the process 
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of the foregoing steps S41 to S46 until 
the user instructs the end of the process 

(S47). The host server apparatus 12 

repeats the process of the foregoing 
steps S51 to S56 until the user instructs 
the end of the process (S57). 

[0065] 

In the operations shown in Figs. 14 
and 15, the host server apparatus 12 
recompresses the image data in adaptive 
manner according to the user position in 
the virtual space. For example, in case 
the user utilizes the interactive 
awareness function, the host server 
apparatus 12 recompresses the image data 
with the large quantization coefficient, 
but, in case of the conference in the 
virtual conference room, the image data 
are transmitted without recompression. 
Therefore, in case of utilizing the 
interactive awareness function, the user 
receives the rough images of other users, 
but, in case of the virtual conference, 
receives the high-definition images of 
other users. In this manner, it is 
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rendered possible to reduce the data 
amount of other users thereby effectively 
utilizing the communication band in case 
of the awareness mode, and to distribute 
the high-definition image in case of 
utilizing the function principally for 
communication such as the conference. 
[ 0 0 6 6] 

The control of the quantization 
coefficient by the server 12 allows 
distributing optimum image data to each 
user. More specifically, it is possible 
to distribute the high-definition image 
data only to the user present in the 
conference room and to distribute the 
highly compressed image, including the 
images of other users present in the 
conference room, to the user present in 
theofficeview. 
[0067] 

The control of the quantization 
coefficient in the server according to 
the position in the virtual space allows 
realizing the optimum image distribution 
while achieving effective utilization of 
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the communication band. 
[0068] 

In the example shown in Fig. 15, 
the image compression rate is changed by 
the host server apparatus 12 according to 
the user position of the transmitting 
terminal, but it is also possible that the 
server apparatus 12 reduces the image 
size in case of the office view but cuts 
out a desired partial image (for example 
a portion of the user face) in other cases. 
Fig. 16 is a flow chart showing the 
operations of the client in such 
variation, and Fig. 17 is a flow chart 
showing the operation of the host server 
apparatus 12. 
[0069] 

The image taken by each video 
camera 42 is taken into the main body 70 
of the computer by a video input device 
thereof, as image data of QCIF format (176 
x 144 pixels) (S61). The fetched image 
data are compression encoded by an image 
encoding method such as JPEG or H.263 
( S 6 2 ) , and are transferred through the 
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network to the host server apparatus 12 

( S 6 3 ) . 

[0070] 

The host server apparatus 12 
receives the image transmitted from the 
client ( S 7 1 ) , and executes partial 
expansion (S72) . For example, in case of 
the image data compressed by the JPEG 
method, the decoding is executed by 
Huffman decoding and inverse DCT to the 
DCT coefficients. Then the host server 
apparatus 12 judges the position of the 
user of the transmitting terminal in the 
virtual space (S73) . 
[0071] 

In case the user of the 
transmitting terminal is present in the 
office view (S73), namely if the user 
utilizes the present system in the 
interactive awareness mode, the image 
size is reduced by deleting a part of the 
DCT coefficients according to the image 
size on the communication channel, in 
order to transmit the full scene showing 
the user state (S74). For example, the 
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reduction is made to the QCIF format (176 
x 144 pixels), corresponding to the image 
shown in Fig. 10(b). 
[0072] 

On the other hand, in case the user 
is present in the conference room (S73), 
an area principally consisting of the 
user face is cut out from the received 
image data (S75). The cut-out area may 
be determined in advance, or may be cut 
out automatically utilizing the face 
recognition technology explained in the 
foregoing. The cut-out image data are 
outputted with the QCIF format, 

corresponding to the image shown in Fig. 
10(c) . 
[0073] 

The size-reduced image (S74) or 
the cut-out image (S75) is quantized, 
then recompressed (Huffman encoding) 
(S76) and distributed to the user 
terminal through the network (S77). 
[0074] 

The user terminal receives the 
images of other users from the host server 
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other users as a rough image in case of 
utilizing the interactive awareness 
function, and the high definition images 
of other users in case of the virtual 
conference. In this manner, it is 

rendered possible to understand the state 
of other users in case of the awareness 
mode, and the images principally composed 
of faces, enabling to understand the 
expression, in case of the function 
mainly for communication such as the 
conference . 
[0077] 

The cutting out of the image by the 
server 12 allows distributing optimum 
image data to each user. More 
specifically, it is possible to 

distribute the image data principally 
composed of the face only to the user 
present in the conference room and to 
distribute the full-scene image, 

including the images of other users 
present in the conference room, to the 
user present in the office view. 
[0078] 
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The control by the server for the area 
or quality of the image to be distributed 
according to the position in the virtual 
space allows realizing the necessary 
image distribution with suitable range 
and quality, while achieving effective 
utilization of the communication band . 
[0079] 

Also the control of the camera 
itself, such as the direction or zooming 
thereof, according to the position in the 
virtual space, is advantageous for 
providing the image of higher quality. 
In distributing for example a magnified 
image, there is required electronic 
enlargement or cut-out if the image 
taking position or the zooming cannot be 
controlled, leading to a lower image 
quality with a reduced number of pixels. 
On the other hand, the control on the image 
taking position or the zooming allows to 
reduce the electronic image processing 
and to provide higher image quality. 
[0080] 

In the foregoing, the JPEG and H.263 
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encoding methods have been cited as the 
examples of the image compression 
encoding method, but such examples are 
not restrictive and there may be employed 
various image compression encoding 
methods . 
[0081] 

Also there have been explained the 
office view corresponding to a virtual 
large space as an example of the space 
utilizing the interactive awareness 
function and the virtual conference room 
as an example of the space realizing the 
active communication, but the present 
invention is not limited to such examples 
and can adapt to various functions. For 
example, there can be conceived a 
training room, a resting room or a library 
virtually constructed on the network, as 
functions for realizing the active 
communication . 
[0 0 8 2 ] 

Also not only to the office space, the 
present invention is applicable to 
communication spaces of other purposes 
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such as a communication tool for a hobby 

group. 

[0083] 

In the foregoing there have been 
explained cases where the operation of 
the user terminal is realized by the 
software on the personal computer. 
However the present invention is not 
limited to such cases. It is for example 
possible to construct the display 
apparatus as a stand-alone exclusive 
terminal and to connect it independently 
to the network. In such case, various 
signal processings and communication 
processes of the user side are all 
executed at such terminal. 
[ 0 0 8 4 ] 

The present invention also 

includes a case where program codes of a 
software realizing the functions of the 
aforementioned embodiments are supplied 
to a computer of an apparatus or a system 
connected to various devices so operating 
such devices as to realize the functions 
of the aforementioned embodiments and the 
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computer (CPU or M P U ) of such apparatus 
of system operate the aforementioned 
devices according to the supplied 
program. 
[0085] 

In such case, the program codes 
themselves of the software realize the 
functions of the aforementioned 

embodiments, and the program codes 
themselves or the means for supplying the 
computer with the program codes, for 
example the memory medium storing the 
program codes, constitute the present 
invention. The memory medium storing the 
program codes can be, for example, a 
floppy disk, a hard disk, an optical disk, 
a magneto-optical disk, a CD-ROM, a 
magnetic tape, a non-volatile memory card 
or a ROM. 
[ 0 0 8 6] 

The present invention naturally 
includes such program codes not only in 
a case where the functions of the 
aforementioned embodiments are realized 
by the execution of the supplied program 
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codes by the computer but also in a case 
where the program codes realize the 
functions of the aforementioned 

embodiments in cooperation with the OS 

(operating system) functioning on the 
computer or other application software. 

[0087] 

Furthermore, the present 

invention includes a case where the 
supplied program codes are once stored in 
a memory provided in a function expansion 
board of the computer or in a function 
expansion unit connected thereto and a 
CPU or the like provided in such function 
expansion board or function expansion 
unit executes all the processes or a part 
thereof according to the instruction of 
the program codes thereby realizing the 
functions of the aforementioned 

embodiments . 
[0088] 

[Effects of the Invention] 

As will be easily understood from 
the foregoing description, the present 
invention can realize optimum image 
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distribution according to the state of 
use of the users, in a virtual office 
system or the like to be utilized in a 
dispersed work environment such as works 
at home or satellite offices. 
[0089] 

Also the present invention allows 
to distribute the necessary image 
information with appropriate area and 
image quality, while effectively 

utilizing the communication band, by 
controlling such area or image quality of 
the image to be distributed at the server 
side. 
[0090] 

Also the control of the camera 
itself, such as the direction or zooming 
thereof, according to the position in the 
virtual space, is advantageous for 
providing the image of higher quality. 

[Brief Description of the Drawings] 

Fig. 1 is a schematic block diagram 
showing the configuration of a first 
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embodiment of the present invention. 

Fig. 2 is a view showing the hardware 
configuration of a host server apparatus 
1 2 . 

Fig. 3 is a view showing the software 
configuration of a host server apparatus 
1 2 . 

Fig. 4 is a view showing the 
configuration of devices in user terminal 
apparatuses 30 to 34. 

Fig. 5 is a view showing an example 
of arrangement of user terminal 

apparatuses 30 to 34. 

Fig. 6 is a view showing an example 
of an image called office view, displayed 
on a virtual office display apparatus 44. 

Fig. 7 is a view showing an example 
of a virtual office display image called 
conference room. 

Fig. 8 is a flow chart showing 
operations relating to obtaining, 

displaying of image data and image 
control in a user terminal apparatus. 

Fig. 9 is a flow chart showing another 
example of the operations of the 
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embodiment . 

Figs. 10 are views showing an example 
of image processing in the operations 
shown in Fig. 9. 

Fig. 11 is a flow chart showing a 3rd 
example of the operations of the 
embodiment . 

Fig. 12 is a plan view showing the 
camera arrangement in case of setting 
plural video cameras. 

Fig. 13 is a flow chart showing the 
operations in case two video cameras are 
set. 

Fig. 14 is a flow chart showing the 
operations of a user terminal in case a 
process according to the user position is 
executed in a host server apparatus 12. 

Fig. 15 is a flow chart showing the 
operations of the host server apparatus 
12 corresponding to Fig. 14. 

Fig. 16 is a flow chart showing the 
operations of a user terminal in another 
example in case the process according to 
the user position is executed in the host 
server apparatus 12. 
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Fig. 17 is a flow chart showing the 
operations of the host server apparatus 
12 corresponding to Fig. 16, and. 

Fig. 18 is a view showing an example 
of a client image in a conventional 
multi-spot video conference system. 

[Description of the Letter Symbols] 

10 Main office 

12 Host server apparatus 

12a BP (Basic Platform) 

12b SPU (Signal Processing Unit) 

14 LAN (Local Area Network) 

16, 18 User terminal apparatus 

20 Computer 

22 User terminal apparatus software 

24 Telephone set 

26 Video camera 

30, 32, 34 User terminal apparatus 

36 Computer 

38 User terminal apparatus software 

40 Telephone set 

42 Video camera 

42a, 42b Video camera 

44 Virtual office display apparatus 
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50 Server manager 

5 2 SPU access library 

54, 56 Driver 

58 Mail transmission unit 

60 DLL (Dynamic Link Library) 

62 Dynamic Web server 

64 Database connector 

66 Windows NT (Trade mark of Microsoft 

Inc., USA) 

53 Database 

70 Main body 

72 Mouse 

74 Keyboard 

76 Display 

78 Microphone 

80 Speaker 

82 Modem (Or a terminal adaptor) 

90 Image showing the work state of 

another user 

92 Work state data display 

94 Private office space 

96 Private office area 

98 Common area 

100 Icon showing conference room A 

102 Icon showing conference room B 
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0 4 Icon showing 

06 Icon showing 

08 Icon showing 

10 Conference r 

12 Image of oth 

14 Image schem 

e s k 

16 Image of a u 

18 Reference di 

20 Video switch 

22 User termina 

24 User 
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[Name of the Document] 
Abstract 
[Abstract ] 
[Subject] 

To provide a user image corresponding 
to the user position inside a virtual 
space 

[Means of Solution] 

User images taken by a video camera 
are fetched and stored in a computer main 
body (SI) . User position inside a virtual 
space is decided ( S 2 ) . In case of an office 
view, a larger quantization coefficient 
is set (S3) and a smaller quantization 
coefficient is set in the other cases, 
also even smaller quantization 

coefficient is set for a face area ( S 4 ) . 
Image data are compression encoded by a 
designated quantization coefficient ( S 5 ) 
and transferred to a host server 
apparatus 12 via a network ( S 6 ) . A user 
terminal receives images of other users 
from a host server apparatus (S7), 
extends the image data ( S 8 ) and displays 
them at a display terminal 4 4 ( S 9 ) . 
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[Preferred Drawing] 
Fig. 8 
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